Physically based rendering of complex scenes can be prohibitively costly with a potentially unbounded and uneven distribution of complexity across the rendered image. The goal of an ideal level of detail (LoD) method is to make rendering costs independent of the 3D scene complexity, while preserving the appearance of the scene. However, current prefiltering LoD methods are limited in the appearances they can support due to their reliance of approximate models and other heuristics. We propose the first comprehensive multi-scale LoD framework for prefiltering 3D environments with complex geometry and materials (e.g., the Disney BRDF), while maintaining the appearance with respect to the ray-traced reference. Using a multi-scale hierarchy of the scene, we perform a data-driven prefiltering step to obtain an appearance phase function and directional coverage mask at each scale. At the heart of our approach is a novel neural representation that encodes this information into a compact latent form that is easy to decode inside a physically based renderer. Once a scene is baked out, our method requires no original geometry, materials, or textures at render time. We demonstrate that our approach compares favorably to state-of-the-art prefiltering methods and achieves considerable savings in memory for complex scenes.
translated by 谷歌翻译
最近的体积三维重建方法可以产生非常精确的结果,即使对于未观察的表面,也可以具有合理的几何形状。然而,当涉及多视图融合时,它们面临着不希望的权衡。它们可以通过全局平均来熔断所有可用视图信息,从而丢失精细的细节,或者他们可以启发式群集对本地融合的群集视图,从而限制他们共同考虑所有视图的能力。我们的关键洞察力是通过在摄像机姿势和图像内容上学习视图融合功能,可以在不限制视图多样性的情况下保留更详细的详细信息。我们建议使用变压器学习此多视图融合。为此,我们使用变压器介绍Vortx,一个端到端的体积3D重建网络,用于宽基线,多视图功能融合。我们的模型是遮挡感知的,利用变压器架构来预测初始投影场景几何估计。该估计用于避免将反射图像特征通过曲面到遮挡区域。我们在Scannet上培训我们的模型,并显示它比最先进的方法产生更好的重建。我们还展示了概括,没有任何微调,优于两个其他数据集,Tum-RGBD和ICL-Nuim的相同最先进的方法。
translated by 谷歌翻译
我们呈现3DVNET,一种新型多视图立体声(MVS)深度预测方法,该方法结合了基于深度和体积的MVS方法的优点。我们的关键思想是使用3D场景建模网络,可迭代地更新一组粗略深度预测,从而产生高度准确的预测,它达成底层场景几何形状。与现有的深度预测技术不同,我们的方法使用体积3D卷积神经网络(CNN),该网络(CNN)在所有深度图中共同地在世界空间上运行。因此,网络可以学习有意义的场景级别。此外,与现有的体积MVS技术不同,我们的3D CNN在特征增强点云上运行,允许有效地聚合多视图信息和灵活的深度映射的迭代细化。实验结果表明,我们的方法超过了Scannet DataSet的深度预测和3D重建度量的最先进的准确性,以及来自Tum-RGBD和ICL-Nuim数据集的一系列场景。这表明我们的方法既有效又推广到新设置。
translated by 谷歌翻译
多模式分类是人类以人为本的机器学习中的核心任务。我们观察到信息跨多模式融合在多模式融合之前,信息在偶像中具有高度互补的信息,因此在多模式融合之前可以彻底稀释。为此,我们呈现稀疏的融合变压器(SFT),一种用于现有最先进的方法的变压器的新型多模式融合方法,同时具有大大降低了内存占用和计算成本。我们想法的关键是稀疏池块,可在跨模式建模之前减少单峰令牌集合。评估在多个多模式基准数据集上进行,用于广泛的分类任务。在类似的实验条件下的多个基准上获得最先进的性能,同时报告计算成本和内存要求降低六倍。广泛的消融研究展示了在天真的方法中结合稀疏和多式化学习的好处。这铺平了在低资源设备上实现多模级学习的方式。
translated by 谷歌翻译
拒绝宇宙射线(CRS)对于CCD捕获数据的科学解释至关重要,但是单曝光图像中检测CRS仍然具有挑战性。传统的CR探测器需要针对不同仪器进行实验参数调整,而最近的深度学习方法仅产生特定于仪器的模型,这些模型遭受了未包括训练数据中未包括的望远镜的性能损失。在这项工作中,我们介绍了宇宙conn,这是在LAS Cumbres天文台(LCO)部署24个望远镜的通用CR探测器。我们首先利用来自LCO的全球望远镜网络的数千张图像来构建一个大型,不同的基于地面的CR数据集,以丰富覆盖仪器和CR功能。然后,我们优化了一个神经网络,并提出了一种新型的CR检测中间加权损耗函数,以训练在LCO成像数据上达到99.91%的真实阳性检测率的通用模型,并在Gemini GMOS-N-n. /s,假阳性率为0.01%。我们还构建了一套工具,包括交互式CR面膜可视化和编辑界面,控制台命令和Python API,以使天文学家社区广泛访问自动,可靠的CR检测。我们的数据集,开源代码库和训练有素的模型可在https://github.com/cy-xu/cosmic-conn上找到。
translated by 谷歌翻译
我们解决了预测动态场景视频的显着图的问题。我们注意到,从固定数量的观察者的凝视数据重建的地图的准确性随帧而变化,因为它取决于场景的内容。当有有限数量的观察者可用时,此问题尤其如此紧迫。在这种情况下,随着传统的深度学习方法,直接最大限度地减少预测和测量的显着性图之间的差异,导致对嘈杂数据过度接受。我们提出了一种噪声感知培训(NAT)范式,这些培训量量化和占帧特异性凝视数据不准确的不确定性。我们表明NAT在有限的培训数据可用时特别有利,在不同模型,丢失函数和数据集中有实验。我们还引入了基于视频游戏的显着数据集,具有富有的时间语义,每帧多个凝视吸引子。数据集和源代码可在https://github.com/nvlabs/nattacy上获得。
translated by 谷歌翻译
In multi-agent systems with large number of agents, typically the contribution of each agent to the value of other agents is minimal (e.g., aggregation systems such as Uber, Deliveroo). In this paper, we consider such multi-agent systems where each agent is self-interested and takes a sequence of decisions and represent them as a Stochastic Non-atomic Congestion Game (SNCG). We derive key properties for equilibrium solutions in SNCG model with non-atomic and also nearly non-atomic agents. With those key equilibrium properties, we provide a novel Multi-Agent Reinforcement Learning (MARL) mechanism that minimizes variance across values of agents in the same state. To demonstrate the utility of this new mechanism, we provide detailed results on a real-world taxi dataset and also a generic simulator for aggregation systems. We show that our approach reduces the variance in revenues earned by taxi drivers, while still providing higher joint revenues than leading approaches.
translated by 谷歌翻译
Object instance segmentation is a key challenge for indoor robots navigating cluttered environments with many small objects. Limitations in 3D sensing capabilities often make it difficult to detect every possible object. While deep learning approaches may be effective for this problem, manually annotating 3D data for supervised learning is time-consuming. In this work, we explore zero-shot instance segmentation (ZSIS) from RGB-D data to identify unseen objects in a semantic category-agnostic manner. We introduce a zero-shot split for Tabletop Objects Dataset (TOD-Z) to enable this study and present a method that uses annotated objects to learn the ``objectness'' of pixels and generalize to unseen object categories in cluttered indoor environments. Our method, SupeRGB-D, groups pixels into small patches based on geometric cues and learns to merge the patches in a deep agglomerative clustering fashion. SupeRGB-D outperforms existing baselines on unseen objects while achieving similar performance on seen objects. Additionally, it is extremely lightweight (0.4 MB memory requirement) and suitable for mobile and robotic applications. The dataset split and code will be made publicly available upon acceptance.
translated by 谷歌翻译
Modern telecom systems are monitored with performance and system logs from multiple application layers and components. Detecting anomalous events from these logs is key to identify security breaches, resource over-utilization, critical/fatal errors, etc. Current supervised log anomaly detection frameworks tend to perform poorly on new types or signatures of anomalies with few or unseen samples in the training data. In this work, we propose a meta-learning-based log anomaly detection framework (LogAnMeta) for detecting anomalies from sequence of log events with few samples. LoganMeta train a hybrid few-shot classifier in an episodic manner. The experimental results demonstrate the efficacy of our proposed method
translated by 谷歌翻译
Opinion mining is the branch of computation that deals with opinions, appraisals, attitudes, and emotions of people and their different aspects. This field has attracted substantial research interest in recent years. Aspect-level (called aspect-based opinion mining) is often desired in practical applications as it provides detailed opinions or sentiments about different aspects of entities and entities themselves, which are usually required for action. Aspect extraction and entity extraction are thus two core tasks of aspect-based opinion mining. his paper has presented a framework of aspect-based opinion mining based on the concept of transfer learning. on real-world customer reviews available on the Amazon website. The model has yielded quite satisfactory results in its task of aspect-based opinion mining.
translated by 谷歌翻译